Image capturing apparatus capable of displaying live view image high in visibility, method of controlling image capturing apparatus, and storage medium

ABSTRACT

An image capturing apparatus capable of displaying a live view image high in visibility on a high-luminance side. An image capturing section converts light from an object to image signals. An image processor performs image processing on image data formed by the image signals. An operation section receives an instruction for setting a live view mode for realizing a live view function. When an OVF simulation mode is set which is different from a recording live view mode for displaying image data subjected to the image processing on the image display section based on user&#39;s photographing settings, photographing is performed under an exposure condition darker than a proper exposure, gradation conversion for compensating for a difference in exposure condition from the proper exposure is performed, and display luminance is controlled to be brighter than display luminance in the recording live view mode.

BACKGROUND OF THE INVENTION Field of the Invention

The present invention relates to an image capturing apparatus that iscapable of displaying a live view image which is high in visibility, amethod of controlling the image capturing apparatus, and a storagemedium.

Description of the Related Art

A mirrorless camera is equipped with, as a function of confirming animage to be photographed, a live view function for displaying a liveview image on an EVF (electronic viewfinder) or a rear liquid crystaldisplay in place of an OVF (optical viewfinder) included in asingle-lens reflex camera. The OVF enables a user to directly view lightincident through an optical lens, and hence the user can view colors andbrightness of an object in the same manner as in a case where the userviews them without using the finder of the camera. Therefore, even whenperforming photographing while following an object, or performingphotographing of a backlight scene or the like, the OVF enables the userto perform photographing while recognizing the object. On the otherhand, the live view image displayed on the EVF or the rear liquidcrystal display is an image generated by performing image processing onan image captured by an image capturing device, and hence the live viewimage is an image in a limited dynamic range e.g. when photographing ofa backlight scene is performed. For this reason, when photographing ascene having a wide dynamic range, such as a backlight scene, it isdifficult to perform photographing while following an object orphotographing while looking at a facial expression of an object, byusing the EVF or the rear liquid crystal display.

In general, in the case of photographing a person in a backlight scene,if photographing is performed with a proper exposure determined based ona person area, a background area is overexposed, and inversely, ifphotographing is performed with a proper exposure determined based onthe background area, the person area becomes dark. Therefore, in thecase of photographing a person in a backlight scene, photographing isperformed under an intermediate exposure between an exposure conditionof a proper exposure for the person area and an exposure condition of aproper exposure for the background area. However, an image photographedunder this exposure condition becomes an image low in visibility inwhich the person is dark and further a high-luminance portion, such as abackground, is small in contrast. In view of such a problem, there is ademand for a technique for enabling an image high in visibility to beacquired even in a backlight scene. As a related art, there has beenproposed a technique in Japanese Laid-Open Patent Publication (Kokai)No. 2017-163339. In Japanese Laid-Open Patent Publication (Kokai) No.2017-163339, for example, gradation conversion is performed on acaptured image so as to obtain linear characteristics over the wholeluminance region from a low-luminance portion to a high-luminanceportion, and the image subjected to this gradation conversion isrecorded. Further, a human eye is high in sensitivity with respect to alow-to-medium luminance region, and hence when this image is displayede.g. as a live view image, gradation conversion for increasing thedisplay luminance in the low-to-medium luminance region is performed onthis image.

However, if the gradation conversion for increasing the displayluminance in the low-to-medium luminance region is performed when theimage is displayed as the live view image as described above, a problemis caused that although the low-to-medium luminance region is adjustedto be easy to be viewed by a human eye, the gradation on ahigh-luminance side is compressed, and hence the visibility on thehigh-luminance side is lowered.

SUMMARY OF THE INVENTION

The present invention provides an image capturing apparatus that iscapable of displaying a live view image which is high in visibility on ahigh-luminance side, a method of controlling the image capturingapparatus, and a storage medium.

In a first aspect of the present invention, there is provided an imagecapturing apparatus, comprising a recording unit, a display unit, atleast one processor, and a memory coupled to the at least one processor,the memory having instructions that, when executed by the processor,perform the operations as: an image capturing unit configured to convertlight from an object to image signals, an image processing unitconfigured to perform image processing on image data formed by the imagesignals obtained by the image capturing unit, and a reception unitconfigured to receive an instruction for setting a live view mode whichis an operation mode for realizing a live view function for displayingimage data subjected to the image processing on the display unit withoutrecording the image data in the recording unit, wherein in a case wherea second live view mode is set which is different from a first live viewmode for displaying image data subjected to the image processing on thedisplay unit based on photographing settings set by a user, the imagecapturing unit performs photographing under an exposure condition in thesecond live view mode, which is set to an exposure condition darker thana determined proper exposure, the image processing unit performsgradation conversion for compensating for a difference in exposurecondition from the proper exposure, and the display unit controlsdisplay luminance in the second live view mode to be brighter thandisplay luminance in the first live view mode.

In a second aspect of the present invention, there is provided a methodof controlling an image capturing apparatus including a recording unit,a display unit, an image capturing unit configured to convert light froman object to image signals, an image processing unit configured toperform image processing on image data formed by the image signalsobtained by the image capturing unit, and a reception unit configured toreceive an instruction for setting a live view mode which is anoperation mode for realizing a live view function for displaying imagedata subjected to the image processing on the display unit withoutrecording the image data in the recording unit, comprising causing, in acase where a second live view mode is set which is different from afirst live view mode for displaying image data subjected to the imageprocessing on the display unit based on photographing settings set by auser, the image capturing unit to perform photographing under anexposure condition in the second live view mode, which is set to anexposure condition darker than a determined proper exposure, the imageprocessing unit to perform gradation conversion for compensating for adifference in exposure condition from the proper exposure, and thedisplay unit to control display luminance in the second live view modeto be brighter than display luminance in the first live view mode.

According to the present invention, the live view image which is high invisibility on the high-luminance side is displayed.

Further features of the present invention will become apparent from thefollowing description of exemplary embodiments (with reference to theattached drawings).

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a schematic block diagram of a digital camera as an imagecapturing apparatus according to an embodiment of the present invention.

FIG. 2 is a block diagram useful in explaining a flow of imageprocessing performed by the digital camera shown in FIG. 1 .

FIG. 3 is a schematic block diagram of an image processor appearing inFIG. 1 .

FIG. 4 is a schematic block diagram of a display conversion sectionappearing in FIG. 3 .

FIG. 5 is a diagram showing an example of maximum display luminancevalues on an image display section appearing in FIG. 1 when a recordinglive view mode and an OVF simulation mode are set, respectively.

FIGS. 6A to 6C are diagrams useful in explaining gradation conversioncharacteristics set to a gradation conversion section appearing in FIG.3 .

FIG. 7 is an image diagram of input/output characteristics of imagesignals, on which scaling is performed by a scaling adjustment sectionappearing in FIG. 4 , and an output luminance.

FIGS. 8A and 8B are diagrams useful in explaining an output luminanceafter gradation conversion performed by an optimization gradationconversion section appearing in FIG. 4 .

FIGS. 9A and 9B are diagrams useful in explaining a display luminancewhen the OVF simulation mode is set.

DESCRIPTION OF THE EMBODIMENTS

The present invention will now be described in detail below withreference to the accompanying drawings showing embodiments thereof

Hereafter, an embodiment of the present invention will be described indetail with reference to the drawings. Note that in the presentembodiment, a description will be given of a case where the presentinvention is applied to a digital camera as an image capturingapparatus, but the present invention is not limitatively applied to thedigital camera. For example, the present invention may be applied to anapparatus equipped with a live view function, such as a personalcomputer (PC), a mobile phone, and a tablet terminal. The digitalcamera, denoted by reference numeral 100, as the image capturingapparatus has a recording mode for performing photographing according tophotographing settings set by a user and recording a photographed image.The photographing settings are recording settings for setting e.g. anexposure correction value, a dynamic range, white balance, color tone,contrast, and a color conversion matrix coefficient, referred tohereinafter. Further, the digital camera 100 is equipped with a liveview function for displaying a live view image on an image displaysection 108, described hereinafter, such as an EVF or a rear liquidcrystal display, based on image signals acquired by an image capturingsection 103, described hereinafter, without recording a photographedimage. As operation modes for realizing the live view function, thedigital camera 100 has a recording live view mode and an OVF simulationmode for simulating an OVF. In the recording live view mode, a live viewimage is displayed on the image display section 108 , according to thesame photographing settings as those used when an image is photographed.In the OVF simulation mode, a live view image adapted to a scene havinga wide dynamic range, such as a backlight scene, is displayed on theimage display section 108. A user sets, when using the live viewfunction, one of the recording live view mode and the OVF simulationmode, using e.g. an operation section 117, described hereinafter.

FIG. 1 is a schematic block diagram of the digital camera 100 as animage capturing apparatus according to the embodiment of the presentinvention.

Referring to FIG. 1 , the digital camera 100 includes a lens group 101,a shutter 102, the image capturing section 103, an analog-to-digitalconverter 104 (simply denoted as “A/D” in FIG. 1 ), an image processor105, an image memory 106, a memory controller 107, the image displaysection 108, a face/facial organ detection section 109, a codec section110, a system controller 111, an interface 112 (simply denoted as “I/F”in FIG. 1 ), a ranging sensor 114, a system memory 115, a nonvolatilememory 116, and the operation section 117 (reception unit).

The lens group 101 is an image capturing optical system including a zoomlens and a focus lens. The shutter 102 includes an aperture function.Light incident through the lens group 101 and the shutter 102 isphotoelectrically converted by the image capturing section 103. Theimage capturing section 103 is implemented e.g. by a CCD or a CMOSdevice, and outputs electrical signals obtained by photoelectricconversion to the analog-to-digital converter 104 as analog imagesignals. The analog-to-digital converter 104 converts the analog imagesignals acquired from the image capturing section 103 to digital imagesignals and outputs the digital image signals to the image processor105.

The image processor 105 performs a variety of image processingoperations, such as color conversion processing including white balance,gradation conversion processing, contour emphasis processing, and colorcorrection processing, on image data formed by digital image signalsacquired from the analog-to-digital converter 104 or image data readfrom the image memory 106 via the memory controller 107. The image datasubjected to image processing is written into the image memory 106 viathe memory controller 107. The image memory 106 stores image data outputfrom the image processor 105 and image data to be displayed on the imagedisplay section 108. The image display section 108 is an EVF, a rearliquid crystal display, or the like.

The face/facial organ detection section 109 detects a face of a person,organs included in a face, such as eyes and a nose, and a facial organarea, from a captured image. The image processor 105 performspredetermined evaluation value calculation processing using a result ofdetection by the face/facial organ detection section 109, a measurementresult obtained by the ranging sensor 114, and the digital image signalsacquired from the image capturing section 103. The system controller 111performs exposure control and ranging control based on an evaluationvalue obtained by this evaluation value calculation processing. Withthis, TTL (through the lens) method-based AF (autofocus) processing, AE(auto exposure) processing, AWB (auto white balance) processing, and soforth are performed. The codec section 110 compresses and encodes imagedata stored in the image memory 106 based on standards, such as JPEG orMPEG.

The system controller 111 controls the overall operation of the digitalcamera 100 by executing programs stored in the system memory 115 or thenonvolatile memory 116. For example, the system controller 111 encodesimage data stored in the image memory 106 and stores the encoded imagedata in a recording medium 113, such as a memory card or a hard disk,via the interface 112. Further, the system controller 111 controls thecodec section 110 to decode and expand image data read out from therecording medium 113 via the interface 112 and stores the image dataprocessed by the codec section 110 in the image memory 106. The imagedata stored in the image memory 106 is displayed on the image displaysection 108. The system memory 115 stores the programs and the like. Thenonvolatile memory 116 stores the programs, setting data, and so forth.The operation section 117 receives a variety of operations performed bya user for giving a setting instruction, a photographing instruction,and so forth. For example, the operation section 117 receives aninstruction for setting one of the recording live view mode and the OVFsimulation mode from the user who uses the live view function.

FIG. 2 is a block diagram useful in explaining a flow of imageprocessing performed by the digital camera 100 shown in FIG. 1 . Analogimage signals output from the image capturing section 103 are convertedto digital image signals by the analog-to-digital converter 104, and thedigital image signals are output to the image processor 105. The imageprocessor 105 generates image data for recording and image data fordisplay based on the acquired digital image signals. The image processor105 outputs the image data for recording to the codec section 110 andoutputs the image data for display to the image display section 108. Theimage data for recording is encoded by the codec section 110 and storedin the recording medium 113. For example, an SDR (Standard DynamicRange) still image is recorded in the recording medium 113 as an imagefile of the JPEG format, and an HDR (High Dynamic Range) still image isrecorded in the recording medium 113 as an image file of the HEIF (HighEfficiency Image File) format. Further, a moving image is recorded inthe recording medium 113 as an image file of e.g. MP4 in common to theSDR image and the HDR image. The image display section 108 displays theimage data for display, acquired from the image processor 105, as thelive view image. The image display section 108 is the EVF or the rearliquid crystal display as mentioned above, which is a device thatreceives an SDR-based sRGB image. The image processor 105 does notoutput the HDR image as it is, but converts the HDR image to an SDRimage and then outputs the SDR image to the image display section 108configured as described above. Further, in the present embodiment, animage can be displayed not only on the image display section 108, butalso on an external display device connected via an external outputterminal, such as an HDMI (registered trademark).

FIG. 3 is a schematic block diagram of the image processor 105 appearingin FIG. 1 . Referring to FIG. 3 , the image processor 105 includes awhite balance multiplying section (denoted as the “WB multiplyingsection” in FIG. 3 ) 301, a color interpolation section 302, a colorconversion section 303, a gradation conversion section 304, athree-dimensional lookup table conversion section 305, a color luminanceconversion section 306, and a display conversion section 307. Processingoperations performed by these components of the image processor 105 arerealized by the system controller 111 that executes programs stored inthe system memory 115 or the nonvolatile memory 116.

The white balance multiplying section 301 multiplies digital imagesignals acquired from the image capturing section 103 (hereinaftersimply referred to as the “image signals”) by white balancecoefficients. The white balance coefficients are auto white balancecoefficients calculated by image analysis for face detection, whitepoint extraction, and the like, or white balance coefficients determinedbased on a preset white balance mode set by the user. The colorinterpolation section 302 performs color interpolation processing on theimage signals processed by the white balance multiplying section 301.RGB signals corresponding to all pixels are generated by colorinterpolation. The color conversion section 303 performs colorconversion processing on image data formed by the image signals aftercolor interpolation. Note that details of the color conversionprocessing will be described hereinafter. The gradation conversionsection 304 sets gradation characteristics and performs gradationconversion on the image data after color conversion processing. Thethree-dimensional lookup table conversion section 305 performs colorconversion on the image data so as to finely adjust color to a desiredcolor tone after gradation conversion, and further, performs gamutmapping suitable for recording and display. The color luminanceconversion section 306 converts image signals forming the image dataprocessed by the three-dimensional lookup table conversion section 305from the RGB signals which are the three primary color signals to YUVsignals which are color luminance signals.

In a case where the user sets the recording mode, the image processor105 outputs the image signals converted to the YUV signals to the codecsection 110. The image data formed by these image signals is encoded bythe codec section 110, and is then recorded in the recording medium 113as an image file of the JPEG format when the SDR image has beenphotographed, and recorded in the recording medium 113 as an image fileof the HEIF format when the HDR image has been photographed.

In a case where the user sets the recording live view mode or the OVFsimulation mode, the display conversion section 307 performs colorconversion and gradation conversion suitable for the image displaysection 108, on the image signals converted to the YUV signals. Then,the image data for display formed by the processed image signals isdisplayed on the image display section 108 as the live view image. Notethat although FIG. 3 shows only one display conversion section 307, theimage processor 105 includes a plurality of display conversion sections307 associated with respective display devices. Each display conversionsection 307 outputs an image suitable for the specifications of anassociated one of the display devices. Thus, in the digital camera 100,the image is displayed on the plurality of display devices at the sametime.

FIG. 4 is a schematic block diagram of the display conversion section307 appearing in FIG. 3 . Referring to FIG. 4 , the display conversionsection 307 includes a YUV-RGB conversion section 401, a scalingadjustment section 402, an inverse gradation conversion section 403, anoptimization gradation conversion section 404, a color space conversionsection 405, a gamut mapping section 406, an output gradation conversionsection 407, and an output gradation adjustment section 408.

The YUV-RGB conversion section 401 converts the image signals convertedto the YUV signals by the color luminance conversion section 306 fromthe YUV signals to RGB signals. The YUV-RGB conversion section 401 usesconversion coefficients associated with an output color space set forthe image processor 105 as conversion coefficients for converting theYUV signals to the RGB signals. The scaling adjustment section 402performs scaling of an output range on the image signals converted tothe RGB signals according to the maximum value which can be assumeddepending on the gradation characteristics set by the gradationconversion section 304.

The inverse gradation conversion section 403 performs inverse gradationconversion on the image signals acquired from the scaling adjustmentsection 402 using EOTF (Electro-Optical Transfer Function) havinginverse gradation conversion characteristics to OETF (Opto-ElectronicTransfer Function) used when the image is generated. In the inversegradation conversion section 403, out of a PQ format and an HLG formatconforming to the ITU-R BT. 2100 standard which is an internationalstandard, the PQ format is employed, for example. The optimizationgradation conversion section 404 performs gradation conversion on theimage signals acquired from the inverse gradation conversion section403, such that display on the image display section 108 is made optimum.The color space conversion section 405 acquires image signals of a Rec.2020 color space from the optimization gradation conversion section 404and converts the color space of the acquired image signals to a colorspace displayed on the image display section 108, such as sRGB. Thegamut mapping section 406 performs gamut mapping processing inaccordance with the image signals acquired from the color spaceconversion section 405 and the color space displayed on the imagedisplay section 108.

The output gradation conversion section 407 performs gradationconversion using the OETF associated with the color space input to theimage display section 108, such as the sRGB, on the image signalsacquired from the gamut mapping section 406. The output gradationadjustment section 408 adjusts the number of gradations of the imagesignals acquired from the output gradation conversion section 407 suchthat the image signals are each caused to have a number of bits whichcan be input to the image display section 108, for example, 8 bitsdefined by the sRGB, and outputs the adjusted image signals to the imagedisplay section 108.

Next, the display luminance characteristics in the image display section108 will be described. The image display section 108 performs gradationconversion on the input signals of 8 bits defined by the sRGB based onthe display luminance setting for controlling the brightness of theimage. In the digital camera 100, when the live view image is displayedin the OVF simulation mode, the display luminance setting is changed toan appropriate value based on the brightness (object luminance) of thesurrounding detected by the digital camera 100.

FIG. 5 is a diagram showing an example of maximum display luminancevalues on the image display section 108 appearing in FIG. 1 when therecording live view mode and the OVF simulation mode are set,respectively. In the digital camera 100, as the display luminancesetting, out of five steps, 1 to 5, of settings, one corresponding tothe object luminance is set. For example, when the setting of thedisplay luminance setting increases by one step, the display luminanceof the image display section 108 becomes 1.75 times brighter. In a casewhere the display luminance setting is set to the same value for therecoding live view mode and the OVF simulation mode, the image displaysection 108 controls the display luminance in the OVF simulation mode tobe 1.75 times brighter, i.e. brighter than the display luminance in therecording live view mode by a value corresponding to one step higher inthe the display luminance setting. Note that in the present embodiment,also when an image recorded in the recording medium 113 in the recordingmode is displayed on the image display section 108, the image isdisplayed on the image display section 108 with the same brightness asin the recording live view mode. That is, in a case where the displayluminance setting is set to the same value for the recoding mode and theOVF simulation mode, the image display section 108 controls the displayluminance in the OVF simulation mode to be 1.75 times brighter, i.e.brighter than the display luminance set when the image recorded in therecording medium 113 is displayed by a value corresponding to one stephigher in the display luminance setting. This control of the displayluminance of the image display section 108 is realized by the systemcontroller 111 that executes an associated program stored in the systemmemory 115 or the nonvolatile memory 116.

Next, the operations of the image capturing section 103 and the colorconversion section 303 performed for image formation by the digitalcamera 100 will be described. The processing operations performed by theimage capturing section 103 and the color conversion section 303 arerealized by the system controller 111 that executes associated programsstored in the system memory 115 or the nonvolatile memory 116.

When the recording mode or the recording live view mode is set, theimage capturing section 103 performs photographing under an exposurecondition determined based on an evaluation result and an analysisresult of the AE processing, and an exposure correction value includedin the photographing settings. On the other hand, when the OVFsimulation mode is set, the image capturing section 103 performsphotographing under an exposure condition one step darker than a properexposure, referred to hereinafter, which has been determined to beproper based on analysis of a scene regardless of the photographingsettings.

The color conversion section 303 sets color conversion matrixcoefficients based on a recording setting included in the photographingsettings when a final image is photographed in the recording mode orwhen an image is captured in the recording live view mode. For example,the color conversion section 303 sets, based on the recording setting,color conversion matrix coefficients adjusted such that colors on whichpreference of a user is reflected are reproduced or color conversionmatrix coefficients adjusted such that faithful colors are reproduced inan output color space. In a case where the output color space is changedby the user, the color conversion matrix coefficients are changedaccording to the output color space specified by the user. In general,an sRGB color space is used as the output color space for the SDR, andthe Rec. 2020 color space is used as the output color space for the HDR.By multiplying the color conversion matrix coefficients adjusted for theSDR, by conversion coefficients for converting the color space from thesRGB color space to the Rec. 2020 color space, the color conversionmatrix coefficients for the HDR can be calculated. In the Rec. 2020color space, a color space wider than the sRGB color space is defined.Therefore, image signals converted by the color conversion matrixcoefficients for the HDR become image signals wider in color gamut thanimage signals converted by the color conversion matrix coefficients forthe SDR. On the other hand, when the OVF simulation mode is set, thecolor conversion section 303 sets color conversion matrix coefficientsfor use in HDR photographing , i.e. the color conversion matrixcoefficients for the HDR. With this, it is possible to display a liveview image in which a scene of a wider color space is reproduced, in theOVF simulation mode.

Next, the gradation conversion characteristics of the gradationconversion section 304 appearing in FIG. 3 will be described. FIG. 6A isa diagram showing examples of gradation characteristics set for thegradation conversion section 304. Reference numeral 601 denotes anexample of gradation characteristics defined by the sRGB. Referencenumeral 602 denotes an example of gradation characteristics set for thegradation conversion section 304 when generating an SDR image, which aregradation characteristics applied when the image is photographed with aproper exposure. Reference numeral 603 denotes an example of gradationcharacteristics set for the gradation conversion section 304 whengenerating an SDR image, which are gradation characteristics set so asto cause photographing to be performed under an exposure condition onestep darker than the proper exposure and increase the luminance level ofthe photographed image to a proper level by gradation conversion. InFIG. 6A, a horizontal axis represents reflectances to associated inputluminance values of an object (input reflectances) expressed by imagesignals input to the gradation conversion section 304, and a verticalaxis represents reflectances (output reflectances) expressed byassociated image signals output from the the gradation conversionsection 304. On the horizontal axis in FIG. 6A, an input reflectance of100% of a white object is expressed as “1.0”, and respective inputreflectances of 200%, 300%, 400%, and 500% are expressed as “2.0”,“3.0”, “4.0”, and “5.0”, respectively.

In general, the exposure condition is determined such that the outputluminance of an object at an input reflectance of 18% becomes apredetermined brightness. Note that the exposure condition may bedetermined at one point in a photographing area, such as in spotphotometry, and further, the exposure condition may be determined basedon information on the whole screen or the brightness of a specificportion in a face area. In the present embodiment, the proper exposurerefers to an exposure under such a condition that the output luminanceof an object at the input reflectance of 18% becomes a predeterminedbrightness. Further, a proper exposure luminance refers to a displayluminance of an image photographed with the propre exposure. In thedigital camera 100, for generating an SDR image and an HDR image,photographing is performed under an exposure condition that the inputreflectance ranges up to 200%, and when the OVF simulation mode is set,photographing is performed under an exposure condition that the inputreflectance ranges up to 400%, by making the image one step darker.Here, the photographing under the exposure condition that the inputreflectance ranges up to 200% refers to photographing performed, bycontrolling the aperture and the like, under an exposure condition thatphotographing can be performed such that, with reference to an inputreflectance of 18% of an object, light up to an input reflectance of200% is not saturated at the image capturing section 103. Note that inthe gradation characteristics in the case of photographing an image witha proper exposure (see e.g. 602), a knee characteristic is set such thatroom for gradation is left on a saturation side by making a dark portioneven darker, so as to make the contrast of an image higher than in thegradation characteristic defined by the sRGB (see e.g. 601).

As described above, photographing is performed under an exposurecondition one step darker than the proper exposure, and gradationcharacteristics for compensating for the difference from the properexposure, more specifically, gradation characteristics for increasingthe luminance level of an image by the above-mentioned exposuredifference, are set for the gradation conversion section 304, whereby itis possible to record image data having output characteristics which aremade higher in the luminance level by one step corresponding to theabove-mentioned exposure difference. However, in the outputcharacteristics, for example, as indicated by 603 in FIG. 6A, thedynamic range is improved in an output reflectance range of not smallerthan 0.8 and the number of gradations in a saturated portion is small,so that the visibility of the saturated portion is low. Therefore, inthe present embodiment, HDR photographing is performed in the OVFsimulation mode.

FIG. 6B is a diagram showing an example of gradation characteristics setfor the gradation conversion section 304 appearing in FIG. 3 . Referencenumeral 604 denotes an example of gradation characteristics of sRGBgamma setting for SDR photographing. Reference numeral 605 denotes anexample of gradation characteristics of PQ gamma setting for HDRphotographing. Reference numeral 606 denotes an example of gradationcharacteristics in the OVF simulation mode. In FIG. 6B, a horizontalaxis represents input reflectances, and a vertical axis representsoutput reflectances. For example, in FIG. 6B, the reference numeral 604indicates gradation characteristics in which an output of 8 bits becomes1.0 on the vertical axis.

An HDR image is recorded in the recording medium 113 as a 10-bit image,and hence the HDR image has gradations which are four times as many asthose of an SDR image recorded as an 8-bit image. In FIG. 6B, thegradation characteristics set for HDR photographing (see e.g. 605) hasthe number of gradations which are larger than that of the gradationcharacteristics set for SDR photographing (see e.g. 604) on ahigh-luminance side. Further, as for the gradation characteristics inthe OVF simulation mode (see e.g. 606), photographing is performed at aninput reflectance of 400% by photographing an object under an exposurecondition one step darker than the proper exposure, and gradationcharacteristics for increasing the luminance level of the image by onestep corresponding to the exposure difference from the proper exposureare set for the gradation conversion section 304, whereby a largernumber of gradations are secured on the high-luminance side than in thecase of gradation characteristic for HDR photographing (see e.g. 605).

FIG. 6C is a diagram showing an example of display luminancecharacteristics appearing when an image converted based on each of thethree types of gradation characteristics indicated in FIG. 6B isdisplayed. In FIG. 6C, a horizontal axis represents input reflectances,and a vertical axis represents display luminance values. The maximumluminance is defined as 100 nits in the sRGB standard, and hence in FIG.6C, reference numeral 607 indicates display luminance characteristicsappearing when the maximum luminance is set to 100 nits. Note that inFIG. 6C, reference numerals 608 and 609 indicate display luminancevalues for HDR photographing and display luminance values in the OVFsimulation mode, which are converted according to the HDR PQ standard,respectively, and the display luminance values are output luminancevalues of an image output to a display monitor conforming to the HDRstandard. In FIG. 6C, comparison between the display luminancecharacteristics for SDR photographing (indicated by 607) and the displayluminance characteristics for HDR photographing (indicated by 608) showsthat the two types of display luminance characteristics are equivalentfrom an input reflectance of 0% (dark portion) to an input reflectanceof 30%. It is found that when the input reflectance exceeds 30%, thedisplay luminance characteristics for HDR photographing have a highergradation property than the display luminance characteristics for SDRphotographing. Further, it is found that the display luminancecharacteristics in the OVF simulation mode (see e.g. 609) has a highergradation property than the display luminance characteristics for HDRphotographing.

Next, the processing performed by the display conversion section 307appearing in FIG. 3 will be described. The scaling adjustment section402 of the display conversion section 307 determines a scalingcoefficient based on the maximum output value of the gradationconversion section 304. In the present embodiment, the scalingadjustment section 402 calculates the scaling coefficient such that theoutput from the inverse gradation conversion section 403 becomes 100nits when in the OVF simulation mode. For example, an output of 100 nitsfor a 10-bit image becomes approximately 520 LSB, from the specificationof the PQ signal defined by ST. 2084, and hence the scaling adjustmentsection 402 calculates the scaling coefficient such that the maximumoutput value of the gradation conversion section 304 becomes 520 LSB.The scaling adjustment section 402 multiplies each signal value of RGBsignals by the calculated scaling coefficient. In the presentembodiment, since photographing is performed in the OVF simulation modeunder the exposure condition one step darker than the proper exposure,the output value of the gradation conversion section 304 becomesapproximately 720 LSB (655 nits after PQ conversion), so that thescaling coefficient is calculated as 520/720=0.72.

FIG. 7 is an image diagram of input/output characteristics of an imagesignal on which scaling has been performed by the scaling adjustmentsection 402 appearing in FIG. 4 , and an output luminance. In FIG. 7 , ahorizontal axis represents input reflectances and a vertical axisrepresents output luminance values. In FIG. 7 , reference numeral 701denotes an example of output luminance values after scaling for SDRphotographing. As indicated by 701, the output luminance value (in nitvalues) after scaling for SDR photographing is set such that the outputluminance value of the image display section 108 at RGB=255 (8 bits)becomes 100 nits. Reference numeral 702 denotes an example of the outputluminance after scaling for HDR photographing. Comparison between 701and 702 shows that in HDR photographing, the gradation property on thehigh-luminance side is higher than in SDR photographing, but darker thanin SDR photographing in a low-to-medium luminance region. That is, inHDR photographing, the visibility in the low-to-medium luminance regionis lower than in SDR photographing. Reference numeral 703 denotes anexample of the output luminance after scaling in the OVF simulationmode. In the OVF simulation mode, as mentioned above, the displayluminance is controlled to be 1.75 times brighter than the displayluminance of the image display section 108 in the recording live viewmode and the display luminance of the image display section 108 when animage recorded in the recording medium 113 in the recording mode isdisplayed. With this, it is possible to improve the visibility of thelive view image in the OVF simulation mode by making the low-to-mediumluminance region bright while maintaining the high gradation property inthe high-luminance side.

FIG. 8A is a diagram showing an example of gradation conversioncharacteristics for display output, which are set for the optimizationgradation conversion section 404 appearing in FIG. 4 . In FIG. 8A, ahorizontal axis represents input luminance values indicated by imagesignals input to the optimization gradation conversion section 404 and avertical axis represents output luminance values of the mage signalsoutput from the optimization gradation conversion section 404. Referencenumeral 800 denotes gradation conversion characteristics for displayoutput when the gamma is 1/1.25. In the present embodiment, the gammavalue to be set for the optimization gradation conversion section 404for adjusting the brightness may be calculated by calculating an outputvalue associated with an input reflectance of 18%, using the outputcharacteristics of SDR and HDR with respect to the input reflectance of18% in the gradation conversion section 304, and the above-describedscaling coefficient. Further, the gradation conversion characteristicsmay be calculated in advance with respect to a combination of anexposure difference between the OVF simulation mode and the properexposure and the brightness setting of the panel.

FIG. 8B is an image diagram of input/output characteristics of an inputreflectance indicated by an image signal and an output luminance aftergradation conversion performed by the optimization gradation conversionsection 404 appearing in FIG. 4 . In FIG. 8B, a horizontal axisrepresents input reflectances, and a vertical axis represents outputluminance values. Reference numeral 801 denotes an example of the outputluminance after gradation conversion performed by the optimizationgradation conversion section 404 for SDR photographing. Referencenumeral 802 denotes an example of the output luminance before gradationconversion performed by the optimization gradation conversion section404 in the OVF simulation mode. Reference numeral 803 denotes an exampleof the output luminance after gradation conversion performed by theoptimization gradation conversion section 404 in the OVF simulationmode. Comparison between 802 and 803 shows that the low-to-mediumluminance region becomes brighter than before gradation conversionperformed by the optimization gradation conversion section 404, and thevisibility of the live view image in the OVF simulation mode is moreimproved.

As described above, according to the present embodiment, in a case wherethe OVF simulation mode is set, the image capturing section 103 performsphotographing by controlling the exposure condition in the OVFsimulation mode to be one step darker than the determined properexposure. The image processor 105 performs gradation conversion forcompensating for the difference in exposure condition from the properexposure. The image display section 108 controls the display luminancein the OVF simulation mode such that the display luminance is madebrighter than the display luminance in the recording live view mode.With this, it is possible to display a live view image in which thegradation on the high-luminance side is not compressed, i.e. a live viewimage which is high in visibility on the high-luminance side.

Further, in the above-described embodiment, when recording an image inthe recording medium 113, an image converted to a 10-bit image isrecorded in the recording medium 113. When displaying an image on theimage display section 108, an image converted to an 8-bit image by theoutput gradation adjustment section 408 is output to the image displaysection 108. Here, according to the HDR standard, the HDR image isdemanded to be recorded in the number of gradations not smaller than 10bits. On the other hand, the display device, such as the EVF, isconfigured to output 8 bits. By taking these into consideration, in thepresent embodiment, an image generating process up to the processingperformed by the color luminance conversion section 306 generates animage in the large number of gradations. When this image is recorded inthe recording medium 113, an image converted to a 10-bit image isrecorded in the recording medium 113. On the other hand, when this imageis displayed on the image display section 108, an image converted to an8-bit image by the output gradation adjustment section 408 is output tothe image display section 108. With this, it is possible to display alive view image which is high in visibility on the high-luminance sideand further, it is possible to record an image having a high gradationproperty in the recording medium 113.

In the above-described embodiment, in the SDR photographing, an 8-Bitimage conforming to the sRGB is generated by the image processor 105.With this, it is possible to reduce the capacity of the image memory 106used for displaying and recording an image.

Further, in the above-described embodiment, in the OVF simulation mode,HDR photographing is set regardless of settings set by a user. Thismakes it possible to acquire image data having a high gradationproperty, and as a result, it is possible to display a live view imagehaving a high gradation property based on this image data.

Further, in the above-described embodiment, the image processor 105includes the plurality of display conversion sections 307 associatedwith the plurality of display devices, respectively. This makes itpossible to provide optimum images for the plurality of display devices,respectively, which are different in specifications. As a result, it ispossible to display a live view image which is high in visibility on thehigh-luminance side on each of the plurality of display devices.

In the above-described embodiment, the display conversion section 307converts the number of gradations of image data to be processed to thenumber of gradations which can be input to an associated display device.This makes it possible to provide each display device with an imagesuited to the specifications of the display device.

In the above-described embodiment, the plurality of display devicesinclude the EVF and the rear liquid crystal display, and hence it ispossible to display a live view image which is high in visibility on thehigh-luminance side on each of the EVF and the rear liquid crystaldisplay.

Although the present invention has been described using theabove-described embodiment, the present invention is not limited to theabove-described embodiment. For example, in the inverse gradationconversion section 403, the HLG format may be employed.

Further, although in the above-described embodiment, in the OVFsimulation mode, the display luminance is controlled to be 1.75 timesbrighter than the display luminance setting in the recording live viewmode, this is not limitative. For example, the display luminance in theOVF simulation mode may be controlled to be equivalent in brightness tothat for SDR photographing in the low-to-medium luminance region. FIG.9A is a diagram showing an example of the maximum display luminancevalues on the image display section 108 appearing in FIG. 1 when thedisplay mode is the recording live view mode and the OVF simulationmode. For example, as shown in FIG. 9A, the display luminance in the OVFsimulation mode is controlled to be four times brighter than the displayluminance setting in the recording live view mode. By this control, forexample, as shown in FIG. 9B, the display luminance in the OVFsimulation mode (see e.g. 901) becomes the same level as the displayluminance for SDR photographing (see e.g. 902) in the low-to-mediumluminance region. With this, it is possible to display a live view imagewhich is high in visibility in the low-to-medium luminance region in theOVF simulation mode without performing gradation conversion by theoptimization gradation conversion section 404.

Further, in the above-described embodiment, photographing may beperformed while displaying a live view image on the image displaysection 108 and outputting an HDR image to an external display deviceconnected via an external output terminal, such as an HDMI. With this,it is possible to display an image which is high in visibility on thehigh luminance side on the external display device which is anHDR-compatible external display device having a larger screen than theimage display section 108.

In the above-described embodiment, the exposure control value in the OVFsimulation mode may be determined according to a dynamic range amountdetermined by scene analysis. In a case where the exposure control valueis automatically determined not based on the photographing settings, butbased on the scene analysis, the gradation conversion characteristics ofthe gradation conversion section 304 or of the output gradationconversion section 407 are adjusted based on the determined exposurecontrol value and the display luminance setting of the image displaysection 108.

Further, in the above-described embodiment, in a case where an operationfor instructing to change the photographing settings is performed by auser during display of the live view image in the OVF simulation mode,an image photographed based on the photographing settings specified bythe user may be recorded in the recording medium 113. For example, in acase where the user sets an exposure correction value during display ofthe live view image in the OVF simulation mode, the live view image onwhich exposure correction based on the set exposure correction value hasnot been performed is displayed as its, but image data photographedbased on this exposure correction value is recorded in the recordingmedium 113.

In the above-described embodiment, the above-mentioned live view imageimproved in visibility on the high-luminance side may be displayed onlyon the EVF, and the live view image corresponding to an image capturedin the recording live view mode, i.e. the live view image based on thesame photographing settings as those used when the image is photographedmay be displayed on the rear liquid crystal display. With this, it ispossible to display a live view image close to one displayed on the OVF,on the EVF which requires a user to look into a small panel, and displaythe image equivalent to the recorded image on the rear liquid crystaldisplay. As a result, the same usage as the single-lens reflex cameracan be realized in a mirrorless camera which is not equipped with theOVF.

Further, in the above-described embodiment, the digital camera 100 maycontrol the maximum luminance of the EVF to be higher than the maximumluminance of the rear liquid crystal display in the OVF simulation mode.With this, a live view image higher in visibility than the display onthe rear liquid crystal display can be displayed on the EVF requiring auser to look into a small panel, and as a result, it is possible toimprove the use feeling of the user.

In the above-described embodiment, when displaying an image recordedafter continuous shooting, the above-mentioned image improved invisibility on the high-luminance side may be displayed on the EVF, andan image obtained according to the photographing settings applied forphotographing may be displayed on the rear liquid crystal display. Bythus controlling the display images, it is possible to display a liveview image which is high in visibility on the EVF during continuousshooting and improve the use feeling of a user.

Note that as described above, the control for displaying differentimages on the EVF and the rear liquid crystal display, respectively, canbe realized by being provided with both of image generating means forgenerating the above-described image improved in visibility on thehigh-luminance side and other image generating means for generating animage based on the photographing settings applied for photographing.

Other Embodiments

Embodiment(s) of the present invention can also be realized by acomputer of a system or apparatus that reads out and executes computerexecutable instructions (e.g., one or more programs) recorded on astorage medium (which may also be referred to more fully as a‘non-transitory computer-readable storage medium’) to perform thefunctions of one or more of the above-described embodiment(s) and/orthat includes one or more circuits (e.g., application specificintegrated circuit (ASIC)) for performing the functions of one or moreof the above-described embodiment(s), and by a method performed by thecomputer of the system or apparatus by, for example, reading out andexecuting the computer executable instructions from the storage mediumto perform the functions of one or more of the above-describedembodiment(s) and/or controlling the one or more circuits to perform thefunctions of one or more of the above-described embodiment(s). Thecomputer may comprise one or more processors (e.g., central processingunit (CPU), micro processing unit (MPU)) and may include a network ofseparate computers or separate processors to read out and execute thecomputer executable instructions. The computer executable instructionsmay be provided to the computer, for example, from a network or thestorage medium. The storage medium may include, for example, one or moreof a hard disk, a random-access memory (RAM), a read only memory (ROM),a storage of distributed computing systems, an optical disk (such as acompact disc (CD), digital versatile disc (DVD), or Blu-ray Disc (BD)™),a flash memory device, a memory card, and the like.

While the present invention has been described with reference toexemplary embodiments, it is to be understood that the invention is notlimited to the disclosed exemplary embodiments. The scope of thefollowing claims is to be accorded the broadest interpretation so as toencompass all such modifications and equivalent structures andfunctions.

This application claims the benefit of Japanese Patent Application No.2021-034886 filed Mar. 5, 2021, which is hereby incorporated byreference herein in its entirety.

What is claimed is:
 1. An image capturing apparatus, comprising: arecording unit; a display unit; at least one processor; and a memorycoupled to the at least one processor, the memory having instructionsthat, when executed by the processor, perform the operations as: animage capturing unit configured to convert light from an object to imagesignals; an image processing unit configured to perform image processingon image data formed by the image signals obtained by the imagecapturing unit; and a reception unit configured to receive aninstruction for setting a live view mode which is an operation mode forrealizing a live view function for displaying image data subjected tothe image processing on the display unit without recording the imagedata in the recording unit, wherein in a case where a second live viewmode is set which is different from a first live view mode fordisplaying image data subjected to the image processing on the displayunit based on photographing settings set by a user, the image capturingunit performs photographing under an exposure condition in the secondlive view mode, which is set to an exposure condition darker than adetermined proper exposure, the image processing unit performs gradationconversion for compensating for a difference in exposure condition fromthe proper exposure, and the display unit controls display luminance inthe second live view mode to be brighter than display luminance in thefirst live view mode.
 2. The image capturing apparatus according toclaim 1, wherein one of SDR (Standard Dynamic Range) photographing andHDR (High Dynamic Range) photographing is set, and wherein in the secondlive view mode, the HDR photographing is set regardless of settings setby the user.
 3. The image capturing apparatus according to claim 2,wherein in a case where the HDR photographing is set, one of a PQ formatand an HLG format conforming to the ITU-R BT. 2100 standard is used. 4.The image capturing apparatus according to claim 2, wherein in a casewhere the HDR photographing is set, the image processing unit executesconversion processing for converting a Rec. 2020 color space to an sRGBcolor space.
 5. The image capturing apparatus according to claim 1,comprising the plurality of display units, and wherein the imageprocessing unit includes a plurality of display conversion unitsassociated with the plurality of display units, respectively.
 6. Theimage capturing apparatus according to claim 5, wherein the displayconversion unit converts the number of gradations of image data to beprocessed to the number of gradations which can be input to anassociated display unit.
 7. The image capturing apparatus according toclaim 5, wherein the plurality of display units include an EVF and arear liquid crystal display.
 8. The image capturing apparatus accordingto claim 7, wherein in the second live view mode, a maximum luminance ofthe EVF is controlled to be higher than a maximum luminance of the rearliquid crystal display.
 9. A method of controlling an image capturingapparatus including a recording unit, a display unit, an image capturingunit configured to convert light from an object to image signals, animage processing unit configured to perform image processing on imagedata formed by the image signals obtained by the image capturing unit,and a reception unit configured to receive an instruction for setting alive view mode which is an operation mode for realizing a live viewfunction for displaying image data subjected to the image processing onthe display unit without recording the image data in the recording unit,comprising: causing, in a case where a second live view mode is setwhich is different from a first live view mode for displaying image datasubjected to the image processing on the display unit based onphotographing settings set by a user, the image capturing unit toperform photographing under an exposure condition in the second liveview mode, which is set to an exposure condition darker than adetermined proper exposure, the image processing unit to performgradation conversion for compensating for a difference in exposurecondition from the proper exposure, and the display unit to controldisplay luminance in the second live view mode to be brighter thandisplay luminance in the first live view mode.
 10. A non-transitorycomputer-readable storage medium storing a program for causing acomputer to execute a method of controlling an image capturing apparatusincluding a recording unit, a display unit, an image capturing unitconfigured to convert light from an object to image signals, an imageprocessing unit configured to perform image processing on image dataformed by the image signals obtained by the image capturing unit, and areception unit configured to receive an instruction for setting a liveview mode which is an operation mode for realizing a live view functionfor displaying image data subjected to the image processing on thedisplay unit without recording the image data in the recording unit,wherein the method comprises: causing, in a case where a second liveview mode is set which is different from a first live view mode fordisplaying image data subjected to the image processing on the displayunit based on photographing settings set by a user, the image capturingunit to perform photographing under an exposure condition in the secondlive view mode, which is set to an exposure condition darker than adetermined proper exposure, the image processing unit to performgradation conversion for compensating for a difference in exposurecondition from the proper exposure, and the display unit to controldisplay luminance in the second live view mode to be brighter thandisplay luminance in the first live view mode.